Search Results: "antonio"

23 November 2015

Lunar: Reproducible builds: week 30 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Mattia Rizzolo uploaded a version of perl to the reproducible repository including the patch written by Niko Tyni to add support for SOURCE_DATE_EPOCH in Pod::Man. Dhole sent an updated version of his patch adding support for SOURCE_DATE_EPOCH in GCC to the upstream mailing list. Several comments have been made in response which have been quickly addressed by Dhole. Dhole also forwarded his patch adding support for SOURCE_DATE_EPOCH in libxslt upstream. Packages fixed The following packages have become reproducible due to changes in their build dependencies: antlr3/3.5.2-3, clusterssh, cme, libdatetime-set-perl, libgraphviz-perl, liblingua-translit-perl, libparse-cpan-packages-perl, libsgmls-perl, license-reconcile, maven-bundle-plugin/2.4.0-2, siggen, stunnel4, systemd, x11proto-kb. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: reproducible.debian.net Vagrant Cascadian has set up a new armhf node using a Raspberry Pi 2. It should soon be added to the Jenkins infrastructure. diffoscope development diffoscope version 42 was release on November 20th. It adds a missing dependency on python3-pkg-resources and to prevent similar regression another autopkgtest to ensure that the command line is functional when Recommends are not installed. Two more encoding related problems have been fixed (#804061, #805418). A missing Build-Depends has also been added on binutils-multiarch to make the test suite pass on architectures other than amd64. Package reviews 180 reviews have been removed, 268 added and 59 updated this week. 70 new fail to build from source bugs have been reported by Chris West, Chris Lamb and Niko Tyni. New issue this week: randomness_in_ocaml_preprocessed_files. Misc. Jim MacArthur started to work on a system to rebuild and compare packages built on reproducible.debian.net using .buildinfo and snapshot.debian.org. On December 1-3rd 2015, a meeting of about 40 participants from 18 different free software projects will be held in Athens, Greece with the intent of improving the collaboration between projects, helping new efforts to be started, and brainstorming on end-user aspects of reproducible builds.

21 September 2015

Lunar: Reproducible builds: week 21 in Stretch cycle

If you see someone on the Debian ReproducibleBuilds project, buy him/her a beer. This work is awesome. What happened in the reproducible builds effort this week: Media coverage Nathan Willis covered our DebConf15 status update in Linux Weekly News. Access to non-LWN subscribers will be given on Thursday 24th. Linux Journal published a more general piece last Tuesday. Unexpected praise for reproducible builds appeared this week in the form of several iOS applications identified as including spyware. The malware was undetected by Apple screening. This actually happened because application developers had simply downloaded a trojaned version of XCode through an unofficial source. While reproducible builds can't really help users of non-free software, this is exactly the kind of attacks that we are trying to prevent in our systems. Toolchain fixes Niko Tyni wrote and uploaded a better patch for the source order problem in libmodule-build-perl. Tristan Seligmann identified how the code generated by python-cffi could be emitted in random order in some cases. Upstream has already fixed the problem. Packages fixed The following 24 packages became reproducible due to changes in their build dependencies: apache-curator, checkbox-ng, gant, gnome-clocks, hawtjni, jackrabbit, jersey1, libjsr305-java, mathjax-docs, mlpy, moap, octave-geometry, paste, pdf.js, pyinotify, pytango, python-asyncssh, python-mock, python-openid, python-repoze.who, shadow, swift, tcpwatch-httpproxy, transfig. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: reproducible.debian.net Tests for Coreboot, OpenWrt, NetBSD, and FreeBSD now runs weekly (instead of monthly). diffoscope development Python 3 offers new features (namely yield from and concurrent.futures) that could help implement parallel processing. The clear separation of bytes and unicode strings is also likely to reduce encoding related issues. Mattia Rizolo thus kicked the effort of porting diffoscope to Python 3. tlsh was the only dependency missing a Python 3 module. This got quickly fixed by a new upload. The rest of the code has been moved to the point where only incompatibilities between Python 2.7 and Pyhon 3.4 had to be changed. The commit stream still require some cleanups but all tests are now passing under Python 3. Documentation update The documentation on how to assemble the weekly reports has been updated. (Lunar) The example on how to use SOURCE_DATE_EPOCH with CMake has been improved. (Ben Beockel, Daniel Kahn Gillmor) The solution for timestamps in man pages generated by Sphinx now uses SOURCE_DATE_EPOCH. (Mattia Rizzolo) Package reviews 45 reviews have been removed, 141 added and 62 updated this week. 67 new FTBFS reports have been filled by Chris Lamb, Niko Tyni, and Lisandro Dami n Nicanor P rez Meyer. New issues added this week: randomness_in_r_rdb_rds_databases, python-ply_compiled_parse_tables. Misc. The prebuilder script is now properly testing umask variations again. Santiago Villa started a discussion on debian-devel on how binNMUs would work for reproducible builds.

1 September 2015

Lunar: Reproducible builds: week 18 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Aur lien Jarno uploaded glibc/2.21-0experimental1 which will fix the issue were locales-all did not behave exactly like locales despite having it in the Provides field. Lunar rebased the pu/reproducible_builds branch for dpkg on top of the released 1.18.2. This made visible an issue with udebs and automatically generated debug packages. The summary from the meeting at DebConf15 between ftpmasters, dpkg mainatainers and reproducible builds folks has been posted to the revelant mailing lists. Packages fixed The following 70 packages became reproducible due to changes in their build dependencies: activemq-activeio, async-http-client, classworlds, clirr, compress-lzf, dbus-c++, felix-bundlerepository, felix-framework, felix-gogo-command, felix-gogo-runtime, felix-gogo-shell, felix-main, felix-shell-tui, felix-shell, findbugs-bcel, gco, gdebi, gecode, geronimo-ejb-3.2-spec, git-repair, gmetric4j, gs-collections, hawtbuf, hawtdispatch, jack-tools, jackson-dataformat-cbor, jackson-dataformat-yaml, jackson-module-jaxb-annotations, jmxetric, json-simple, kryo-serializers, lhapdf, libccrtp, libclaw, libcommoncpp2, libftdi1, libjboss-marshalling-java, libmimic, libphysfs, libxstream-java, limereg, maven-debian-helper, maven-filtering, maven-invoker, mochiweb, mongo-java-driver, mqtt-client, netty-3.9, openhft-chronicle-queue, openhft-compiler, openhft-lang, pavucontrol, plexus-ant-factory, plexus-archiver, plexus-bsh-factory, plexus-cdc, plexus-classworlds2, plexus-component-metadata, plexus-container-default, plexus-io, pytone, scolasync, sisu-ioc, snappy-java, spatial4j-0.4, tika, treeline, wss4j, xtalk, zshdb. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: Chris Lamb also noticed that binaries shipped with libsilo-bin did not work. Documentation update Chris Lamb and Ximin Luo assembled a proper specification for SOURCE_DATE_EPOCH in the hope to convince more upstreams to adopt it. Thanks to Holger it is published under a non-Debian domain name. Lunar documented easiest way to solve issues with file ordering and timestamps in tarballs that came with tar/1.28-1. Some examples on how to use SOURCE_DATE_EPOCH have been improved to support systems without GNU date. reproducible.debian.net armhf is finally being tested, which also means the remote building of Debian packages finally works! This paves the way to perform the tests on even more architectures and doing variations on CPU and date. Some packages even produce the same binary Arch:all packages on different architectures (1, 2). (h01ger) Tests for FreeBSD are finally running. (h01ger) As it seems the gcc5 transition has cooled off, we schedule sid more often than testing again on amd64. (h01ger) disorderfs has been built and installed on all build nodes (amd64 and armhf). One issue related to permissions for root and unpriviliged users needs to be solved before disorderfs can be used on reproducible.debian.net. (h01ger) strip-nondeterminism Version 0.011-1 has been released on August 29th. The new version updates dh_strip_nondeterminism to match recent changes in debhelper. (Andrew Ayer) disorderfs disorderfs, the new FUSE filesystem to ease testing of filesystem-related variations, is now almost ready to be used. Version 0.2.0 adds support for extended attributes. Since then Andrew Ayer also added support to reverse directory entries instead of shuffling them, and arbitrary padding to the number of blocks used by files. Package reviews 142 reviews have been removed, 48 added and 259 updated this week. Santiago Vila renamed the not_using_dh_builddeb issue into varying_mtimes_in_data_tar_gz_or_control_tar_gz to align better with other tag names. New issue identified this week: random_order_in_python_doit_completion. 37 FTBFS issues have been reported by Chris West (Faux) and Chris Lamb. Misc. h01ger gave a talk at FrOSCon on August 23rd. Recordings are already online. These reports are being reviewed and enhanced every week by many people hanging out on #debian-reproducible. Huge thanks!

30 August 2015

Antonio Terceiro: DebConf15, testing debian packages, and packaging the free software web

This is my August update, and by the far the coolest thing in it is Debconf. Debconf15 I don t get tired of saying it is the best conference I ever attended. First it s a mix of meeting both new people and old friends, having the chance to chat with people whose work you admire but never had a chance to meet before. Second, it s always quality time: an informal environment, interesting and constructive presentations and discussions. This year the venue was again very nice. Another thing that was very nice was having so many kids and families. This was no coincidence, since this was the first DebConf in which there was organized childcare. As the community gets older, this a very good way of keeping those who start having kids from being alienated from the community. Of course, not being a parent yet I have no idea how actually hard is to bring small kids to a conference like DebConf. ;-) I presented two talks: There was also the now traditional Ruby BoF, where discussed the state and future of the Ruby ecosystem in Debian; and an in promptu Ruby packaging workshop where we introduced the basics of packaging in general, and Ruby packaging specifically. Besides shak, I was able to hack on a few cool things during DebConf: Miscellaneous updates

25 August 2015

Lunar: Reproducible builds: week 17 in Stretch cycle

A good amount of the Debian reproducible builds team had the chance to enjoy face-to-face interactions during DebConf15.
Names in red and blue were all present at DebConf15
Picture of the  reproducible builds  talk during DebConf15
Hugging people with whom one has been working tirelessly for months gives a lot of warm-fuzzy feelings. Several recorded and hallway discussions paved the way to solve the remaining issues to get reproducible builds part of Debian proper. Both talks from the Debian Project Leader and the release team mentioned the effort as important for the future of Debian. A forty-five minutes talk presented the state of the reproducible builds effort. It was then followed by an hour long roundtable to discuss current blockers regarding dpkg, .buildinfo and their integration in the archive. Picture of the  reproducible builds  roundtable during DebConf15 Toolchain fixes Reiner Herrmann submitted a patch to make rdfind sort the processed files before doing any operation. Chris Lamb proposed a new patch for wheel implementing support for SOURCE_DATE_EPOCH instead of the custom WHEEL_FORCE_TIMESTAMP. akira sent one making man2html SOURCE_DATE_EPOCH aware. St phane Glondu reported that dpkg-source would not respect tarball permissions when unpacking under a umask of 002. After hours of iterative testing during the DebConf workshop, Sandro Knau created a test case showing how pdflatex output can be non-deterministic with some PNG files. Packages fixed The following 65 packages became reproducible due to changes in their build dependencies: alacarte, arbtt, bullet, ccfits, commons-daemon, crack-attack, d-conf, ejabberd-contrib, erlang-bear, erlang-cherly, erlang-cowlib, erlang-folsom, erlang-goldrush, erlang-ibrowse, erlang-jiffy, erlang-lager, erlang-lhttpc, erlang-meck, erlang-p1-cache-tab, erlang-p1-iconv, erlang-p1-logger, erlang-p1-mysql, erlang-p1-pam, erlang-p1-pgsql, erlang-p1-sip, erlang-p1-stringprep, erlang-p1-stun, erlang-p1-tls, erlang-p1-utils, erlang-p1-xml, erlang-p1-yaml, erlang-p1-zlib, erlang-ranch, erlang-redis-client, erlang-uuid, freecontact, givaro, glade, gnome-shell, gupnp, gvfs, htseq, jags, jana, knot, libconfig, libkolab, libmatio, libvsqlitepp, mpmath, octave-zenity, openigtlink, paman, pisa, pynifti, qof, ruby-blankslate, ruby-xml-simple, timingframework, trace-cmd, tsung, wings3d, xdg-user-dirs, xz-utils, zpspell. The following packages became reproducible after getting fixed: Uploads that might have fixed reproducibility issues: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: St phane Glondu reported two issues regarding embedded build date in omake and cduce. Aur lien Jarno submitted a fix for the breakage of make-dfsg test suite. As binutils now creates deterministic libraries by default, Aur lien's patch makes use of a wrapper to give the U flag to ar. Reiner Herrmann reported an issue with pound which embeds random dhparams in its code during the build. Better solutions are yet to be found. reproducible.debian.net Package pages on reproducible.debian.net now have a new layout improving readability designed by Mattia Rizzolo, h01ger, and Ulrike. The navigation is now on the left as vertical space is more valuable nowadays. armhf is now enabled on all pages except the dashboard. Actual tests on armhf are expected to start shortly. (Mattia Rizzolo, h01ger) The limit on how many packages people can schedule using the reschedule script on Alioth has been bumped to 200. (h01ger) mod_rewrite is now used instead of JavaScript for the form in the dashboard. (h01ger) Following the rename of the software, debbindiff has mostly been replaced by either diffoscope or differences in generated HTML and IRC notification output. Connections to UDD have been made more robust. (Mattia Rizzolo) diffoscope development diffoscope version 31 was released on August 21st. This version improves fuzzy-matching by using the tlsh algorithm instead of ssdeep. New command line options are available: --max-diff-input-lines and --max-diff-block-lines to override limits on diff input and output (Reiner Herrmann), --debugger to dump the user into pdb in case of crashes (Mattia Rizzolo). jar archives should now be detected properly (Reiner Herrman). Several general code cleanups were also done by Chris Lamb. strip-nondeterminism development Andrew Ayer released strip-nondeterminism version 0.010-1. Java properties file in jar should now be detected more accurately. A missing dependency spotted by St phane Glondu has been added. Testing directory ordering issues: disorderfs During the reproducible builds workshop at DebConf, participants identified that we were still short of a good way to test variations on filesystem behaviors (e.g. file ordering or disk usage). Andrew Ayer took a couple of hours to create disorderfs. Based on FUSE, disorderfs in an overlay filesystem that will mount the content of a directory at another location. For this first version, it will make the order in which files appear in a directory random. Documentation update Dhole documented how to implement support for SOURCE_DATE_EPOCH in Python, bash, Makefiles, CMake, and C. Chris Lamb started to convert the wiki page describing SOURCE_DATE_EPOCH into a Freedesktop-like specification in the hope that it will convince more upstream to adopt it. Package reviews 44 reviews have been removed, 192 added and 77 updated this week. New issues identified this week: locale_dependent_order_in_devlibs_depends, randomness_in_ocaml_startup_files, randomness_in_ocaml_packed_libraries, randomness_in_ocaml_custom_executables, undeterministic_symlinking_by_rdfind, random_build_path_by_golang_compiler, and images_in_pdf_generated_by_latex. 117 new FTBFS bugs have been reported by Chris Lamb, Chris West (Faux), and Niko Tyni. Misc. Some reproducibility issues might face us very late. Chris Lamb noticed that the test suite for python-pykmip was now failing because its test certificates have expired. Let's hope no packages are hiding a certificate valid for 10 years somewhere in their source! Pictures courtesy and copyright of Debian's own paparazzi: Aigars Mahinovs.

16 August 2015

Lunar: Reproducible builds: week 16 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Valentin Lorentz sent a patch for ispell to initialize memory structures before dumping their content. In our experimental repository, qt4-x11 has been rebased on the latest version (Dhole), as was doxygen (akira). Packages fixed The following packages became reproducible due to changes in their build dependencies: backup-manager, cheese, coinor-csdp, coinor-dylp, ebook-speaker, freefem, indent, libjbcrypt-java, qtquick1-opensource-src, ruby-coffee-script, ruby-distribution, schroot, twittering-mode. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: akira found another embedded code copy of texi2html in maxima. reproducible.debian.net Work on testing several architectures has continued. (Mattia/h01ger) Package reviews 29 reviews have been removed, 187 added and 34 updated this week. 172 new FTBFS reports were filled, 137 solely by Chris West (Faux). josch spent time investigating the issue with fonts in PDF files. Chris Lamb documented the issue affecting documentation generated by ocamldoc. Misc. Lunar presented a general Reproducible builds HOWTO talk at the Chaos Communication Camp 2015 in Germany on August 13th. Recordings are already available, as well as slides and script. h01ger and Lunar also used CCCamp15 as an opportunity to have discussions with members of several different projects about reproducible builds. Good news should be coming soon.

5 August 2015

Antonio Terceiro: Elixir in Debian, MiniDebconf at FISL, and Debian CI updates

In June I started keeping track of my Debian activities, and this is my July update. Elixir in Debian Elixir is a functional language built on top of the Erlang virtual machine. If features imutable data structures, interesting concurrency primitives, and everything else that Erlang does, but with a syntax inspired by Ruby what makes it much more aproachable in my opinion. Those interested in Elixir for Debian are encouraged to hang around in #debian-elixir on the OFTC IRC servers. There are still a lot of things to figure out, for example how packaging Elixir libraries and applications is going to work. MiniDebconf at FISL, and beyond I helped organize a MiniDebconf at this year s FISL, in Porto Alegre on the 10th of July. The whole program was targetted at getting more people to participate in Debian, so there were talks about translation, packaging, and a few other more specific topics. I myself gave two talks: one about Debian basics, What is Debian, and how it works , and second one on packaging the free software web , which I will also give at Debconf15 later this month. The recordings are available (all talks in Portuguese) at the Debian video archive thanks to Holger Levsen. We are also organizing a new MiniDebconf in October as part of the Latinoware schedule. Ruby We are in the middle of a transition to switch to Ruby 2.2 as default in Debian unstable, and we are almost there. The Ruby transition is now on hold while GCC 5 one is going on, but will be picked up as soon as were are done with GCC 5. ruby-defaults has been uploaded to experimental for those that want to try having Ruby 2.2 as default before that change hits unstable. I myself have been using Ruby 2.2 as default for several weeks without any problem so far, including using vagrant on a daily basis and doing all my development on sid with it. I started taking notes about Ruby interpreter transitions work to make sure that knowledge is registered. I have uploaded minor security updates of both ruby2.1 and ruby2.2 to unstable. They both reached testing earlier today. I have also fixed another bug in redmine, which I hope to get into stable as well as soon as possible. gem2deb has seen several improvements through versions 0.19, 0.20, 0.20.1 and 0.20.2. I have updated a few packages: Two NEW packages, ruby-rack-contrib and ruby-grape-logging ,were ACCEPTED into the Debian archive. Kudos to the ftp-master team who are doing an awesome job reviewing new packages really fast. Debian Continuous Integration This month I have made good progress with the changes needed to make debci work as a distributed system with one master/scheduler node and as many worker nodes (running tests) as possible. While doing my tests, I have submitted a patch to lxc and updated autodep8 in unstable. At some point I plan to upload both autodep8 and autopkgtest to jessie-backports. Sponsoring I have sponsored a few packages:

26 July 2015

Lunar: Reproducible builds: week 13 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes akira uploaded a new version of doxygen in the experimental reproducible repository incorporating upstream patch for SOURCE_DATE_EPOCH, and now producing timezone independent timestamps. Dhole updated Peter De Wachter's patch on ghostscript to use SOURCE_DATE_EPOCH and use UTC as a timezone. A modified package is now being experimented. Packages fixed The following 14 packages became reproducible due to changes in their build dependencies: bino, cfengine2, fwknop, gnome-software, jnr-constants, libextractor, libgtop2, maven-compiler-plugin, mk-configure, nanoc, octave-splines, octave-symbolic, riece, vdr-plugin-infosatepg. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: reproducible.debian.net Packages identified as failing to build from source with no bugs filed and older than 10 days are scheduled more often now (except in experimental). (h01ger) Package reviews 178 obsolete reviews have been removed, 59 added and 122 updated this week. New issue identified this week: random_order_in_ruby_rdoc_indices. 18 new bugs for packages failing to build from sources have been reported by Chris West (Faux), and h01ger.

2 July 2015

Antonio Terceiro: Upgrades to Jessie, Ruby 2.2 transition, and chef update

Last month I started to track all the small Debian-related things that I do. My initial motivation was to be concious about how often I spend short periods of time working on Debian. Sometimes it s during lunch breaks, weekends, first thing in the morning before regular work, after I am done for the day with regular work, or even during regular work, since I do have the chance of doing Debian work as part of my regular work occasionally. Now that I have this information, I need to do something with it. So this is probably the first of monthly updates I will post about my Debian work. Hopefully it won t be the last. Upgrades to Jessie I (finally) upgraded my two servers to Jessie. The first one, my home server, is a Utilite which is a quite nice ARM box. It is silent and consumes very little power. The only problem I had with it is that the vendor-provided kernel is too old, so I couldn t upgrade udev, and therefore couldn t switch to systemd. I had to force systemv for now, until I can manage to upgrade the kernel and configure uboot to properly boot the official Debian kernel. On my VPS things are way better. I was able to upgrade nicely, and it is now running a stock Jessie system. fixed https on ci.debian.net pabs had let me know on IRC of an issue with the TLS certificate for ci.debian.net, which took me a few iterations to get right. It was missing the intermediate certificates, and is now fixed. You can now enjoy Debian CI under https . Ruby 2.2 transition I was able to start the Ruby 2.2 transition, which has the goal of switch to Ruby 2.2 on unstable. The first step was updating ruby-defaults adding support to build Ruby packgaes for both Ruby 2.1 and Ruby 2.2. This was followed by updates to gem2deb (0.18, 0.18.1, 0.18.2, and 0.18.3) and rubygems-integration . At this point, after a few rebuild requests only 50 out of 137 packages need to be looked at; some of them just use the default Ruby, so a rebuild once we switch the default will be enough to make it use Ruby 2.2, while others, specially Ruby libraries, will still need porting work or other fixes. Updated the Chef stack Bringing chef to the very latest upstream release into unstable was quite some work. I had to update: In the middle I also had to package a new dependency, ruby-ffi-yajl, which was very quickly ACCEPTED thanks to the awesome work of the ftp-master team. Random bits

9 June 2015

Tiago Bortoletto Vaz: Zyne is now in Debian

Zyne is a modular synthetizer written in Python. Anyone can create and extend its modules using the Pyo library. Zyne's GUI is coded using WXPython and will look nicely in GNU/Linux, Mac and Windows systems. It's written by the same author of Pyo, and together with Cecilia and Soundgrain is part of an amazing set of libre tools for sound synthesis and electronic music composition.
/images/zyne-screenshot.png

Zyne loading 6 of its 14 default modules

Zyne package is result of a successful one-day event called MicroDebconf Brasilia 2015, being created during a track about packaging and QA leaded by Eriberto Mota and Antonio Terceiro.

11 May 2015

Bits from Debian: Debian Ruby team sprint 2015

The Debian Ruby Ruby team had a first sprint in 2014. The experience was very positive, and it was decided to do it again in 2015. Last April, the team once more met at the IRILL offices, in Paris, France. The participants worked to improve the quality Ruby packages in Debian, including fixing release critical and security bugs, improving metadata and packaging code, and triaging test failures on the Debian Continuous Integration service. The sprint also served to prepare the team infrastructure for the future Debian 9 release: Group photo of sprint participants. Left to right: Christian Hofstaedtler, Tomasz Nitecki, Sebastien Badia and Antonio Terceiro Left to right: Christian Hofstaedtler, Tomasz Nitecki, Sebastien Badia and Antonio Terceiro. A full report with technical details has been posted to the relevant Debian mailing lists.

20 March 2015

Antonio Terceiro: rrg: visualize the require hierarchy in Ruby projects

Yesterday I was hacking on some Ruby code and getting a weird error which I thought was caused by mutually recursive require statements (i.e. A requires B, and B requires A). Later I realized that this is not an issue in Ruby, since the intepreter keeps track of what has already been required and will not enter a loop. But during the investigation I came up with something that turned out to be useful. rrg will read the source code of a Ruby project and generate a graph based on the require statements in the code; nodes represent the source files and an arrow from A to B means that A contains a require B statement. From the README:
Just run rrg at the root of your project. rrg will parse the code inside lib/, and generate a graph description in the Graphviz format. You can pipe the output to Graphviz directly, or store it in a file and process it to generate an image.
If you call rrgv instead, it will automatically process the graph with Graphviz, generate a PNG image, and open it.
Let s see some examples. First the classical analysing itself example, the require graph for rrg itself: Not very interesting, since all of the logic is currently in the main binary and not on library code. But 1) when I do the refactorings I want to, there will be more library code and 2) while writing this I implemented also parsing scripts in bin/. Now chake which is a slightly larger project: An even larger (but still not that big) project, gem2deb: Note that these visualizations may not be accurate representations of the actual source code. In Ruby, nothing stops one from implementing class A::B in lib/x/y.rb, but most reasonable code will make sure that filenames and the classes namespaces actually match. If you are working on a sane codebase, though, visualizing graphs like this helps understand the general structure of the code and perceive possible improvements. The gem2deb graph gave me some ideas already and I didn t even paid much attention to it yet.

4 March 2015

Zlatan Todori : Interviews with FLOSS developers: Paul Wise

After starting with Joey Hess, we continue with Paul Wise. What makes his star to shine are many things such as being a DSA (Debian System Administrator), a helpful hand on mailings list, encouraging people to join Debian teams but most of all - he has encyclopedia knowledge on Debian as a whole which he gladly shares with anyone who asks (very fast response on IRC channels). It is almost impossible for any single person to count all Debian teams, work and places - to know most of those things, you can image the vast knowledge which Paul has. The legend says that his brain has better and faster search engine algorithm on Debian related queries than all other engines combined. So lets see what he has to share with world. me: Who are you? pabs: Paul Wise (pabs) and I have to say that I'm no-where near as knowledgeable as your intro suggests. me: How did you start programming? pabs: Messing around with fractals and graphics things in MS BASIC. me: How would you now advise others to start programming? pabs: Pick an issue in a tool you use, investigate how the tool works and how you can change it, fix that and contribute the change back to the project that created that tool. In the process you will learn skills, interact with the community and contribute to the project. me: Setup of your development machine? pabs: Lenovo Thinkpad with external monitor, Debian testing and some tweaks me What is your preferable language (for hacking)? Why? How do you compare it to other languages? pabs: I currently prefer Python for its readability. It still has some rough edges though the documentation covers them fairly well. I generally pick up new languages when working on projects written in them. Haskell is next on the horizon due to Nikki and the Robots. me: Describe your current most memorable situation as software developer/hacker? pabs: I had a great time creating fractals in BASIC, learning about the Mandelbrot set, L-systems and more. My days and nights of hacking on frhed (a GPLed hex editor for Windows) to help me cheat at Civilisation were pretty memorable. frhed led to my work on reverse engineering the CHM file format (a documentation format for Windows programs). A stand-out moment during my time with Debian was hacking on the derivates census patch generation code during the Debian UK BBQ weekend, surrounded by geeks playing Portal, cooking things, hacking on Debian and generally having a good time (thanks Steve!). me: Some memorable moments from Debian conferences? pabs: There are so many; meeting Debian folks, playing Mao once and then never again, late night games of werewolf, both delectably delicious and hideously disgusting cheeses, fried insects, day trips to beautiful landscapes, inspiring keynotes, exciting BoFs, secret IRC channels for planning surprise birthday parties, blue hair, wet air, blocks of fried cheese, a vast quantity of icecream, pants, geeks in the surf, volcanoes, hiking, a wonderful view, a uni-cycling stormtrooper & more. me: How do you see future of Debian development? pabs: I hope we will continue to exist and uphold our principles for the foreseeable future. I don't have any crystal balls though. me: You recently became member of Debian DSA - what is that like, what roles do you have and what tasks are in front of DSA? pabs: We wrote a bit of text about that for DPN recently. me: You have large knowledge on Debian and you share it with anyone who wants to know more. What motivates you to do so? pabs: I want the operating system I personally rely on to exist into the future, helping folks work on and join Debian can help with that. me: Why should developers and users join Debian community? What makes Debian a great and happy place? pabs: Every Debian contributor has different reasons for joining the community. Personally the Social Contract, the DFSG and the spirit and culture behind them are the main reason to be involved. I also like our many efforts towards technical excellence and correctness. Of course I've made a number of good friends over the years, especially as a result of attending DebConf every year since 2007. me: You are member of Debian publicity team which writes Debian news - do you need more people to join that team and how can they start? pabs: Since there is an infinite amount of work to do, pretty much every part of Debian always needs help, that includes the publicity team. We published a post about ways to help here. me: If someone wants to contribute to Debian in terms of packaging, can they do it anonymously (for example over Tor network, does Debian have .onion address)? pabs: Due to Debian's penchant for transparency it is harder but there are definitely package maintainers who have built up a reputation for good work under a pseudonym over the years and become Debian contributors as a result. I'm not aware of completely anonymous package maintainers but there are definitely people who file bugs using one-off pseudonyms, which is almost the same thing as anonymously. There are definitely Debian contributors and members who use Tor while contributing to Debian. In fact, as Debian is very highly dependent on OpenPGP and the best practices for OpenPGP include refreshing your keyring slowly over Tor, so probably quite a number of Debian contributors use Tor. As far as I know Debian itself does not run any Tor relays or onion services. me: What are places that non-packaging developers and people could join and help spread Debian even more? pabs: There are many ways to help Debian, including non-technical ones. Unfortunately our web page about helping Debian isn't quite up-to-date with all of them but a few more are to volunteer at DebConf, helo with artwork requests, speak about Debian at events or even come up with ideas for projects. Whatever skills you have, Debian can probably make use of them. If you aren't sure where to start, jump on the debian-mentors mailing list or IRC channel and we can probably guide you to the right place within Debian. Don't worry about not being skilled enough, everyone starts somewhere. me: How do you see Debian will manage webapps? pabs: Personally I prefer locally installed software, standard data formats and standard data transfer protocols to the wild webapps world but I understand they are becoming very popular to produce and use due to the ubiquity of the web browser platform. Antonio Terceiro is mentoring a project for this year's newcomer mentorship programs (outreachy/gsoc) that aims to improve support for installing web apps on Debian installations. I hope it succeeds as it could help make Debian more popular on servers and home servers in particular. me: How would you advise Debian (and other FLOSS users) to setup their machine in terms of security and anonymity? pabs: All technology has upsides and downsides. I would advise anyone to analyse their situation and protect themselves accordingly. For example if you have a bad memory, full disk encryption, which is based on pass-phrases might lead to data loss and physical security might be a better choice for protecting your data. The right choices around technology are very much a personal thing. me: Is it better to setup xmonad (because it is Haskell based WM) with small dependency chain or GNOME (because it is getting sandboxed apps) in term of security and privacy implications? pabs: Again, the right choices around technology are very much a personal thing. Due to the design of X11, both of these are approximately equivalent from a window-manager security properties point of view, that is to say, pretty bad. Wayland is one of the possible X11 successors and offers much better security properties. GNOME folks are working on switching to Wayland. Ultimately though it comes down to how each person uses their window manager and which software they run under it. me: Should Debian join Tor project as distro that installs Tor relays by default - should it offer that as option in installer in Debian 9? pabs: Running a Tor relay requires a reasonably fast and reliable Internet connection and should be a conscious decision on behalf of the sysadmin for a computer so Debian probably shouldn't install them by default. If tasksel gets support for installing tasks from Debian Pure Blends, then we could add a Tor relay task to the Debian Sanctuary Pure Blend. me: Have you ever considered joining initiatives such as FreedomBox? pabs: I was quite moved by Eben Moglen's talk at DebConf10 in New York and the resulting BoF. It seemed like a very ambitious project but I didn't really have the knowledge, skills or time to contribute yet. me: Are you a gamer? Valve Steam games are offered for free to Debian Developers - do you use steam and play Valve games? Your thoughts on Steam and non-free Linux gaming? pabs: I play computer games occasionally, all from Debian main or ones that I'm packaging. 0ad is my current go-to for a bit of gaming. I don't have any experience with Steam or non-free games on Linux. me: Is there something you would change in FLOSS ecosystem? pabs: Various folks have highlighted new and ongoing challenges for the FLOSS ecosystem in various places in recent years. Something that I would like to highlight that does not get talked about enough is the choices we make around our digital artefacts. This is the discussion around "preferred form for modification" or "source". The "source" for a particular digital artefact is a deliberate choice on behalf of the authors. Often generated files are distributed alongside the "source" without any instructions for reproducing the generated files from the "source". It sometimes happens that FLOSS contributors forget to distriute what they have chosen as "source", instead just distributing the generated files. This is a fairly well known issue but still happens. What isn't thought about quite as much is that the choice of "source" has consequences for future development possibilities of that "source". Some forms of "source" are more expressive than others, can be modified in a wider variety of ways and are better choices in general. Sometimes the consequences of choosing less expressive forms are mild and other times they are quite important. I hope more people will start to think about these choices. Some examples where, in my opinion, various people could have made better choices are listed in the mail I sent to the games team list last year. Another thing I would like to highlight is the work that organisations like Software Freedom Conservancy and Software in the Public Interest do to protect, defend, promote and support FLOSS projects. It is very important work that needs our interest and support. me: Can FLOSS world create great alternatives to Viber, Dropbox, WhatsUp, Facebook, Skype and other non-free services? pabs: I think that the FLOSS world has already created alternatives to all of those. The success of non-free services doesn't take these alternatives away but it does mean some of them are less useful because some of them are the kind of tools that become more useful with a larger amount of people using them. I don't know what it would take for the FLOSS alternatives to achieve similar success as network effects are hard to overcome. Hopefully mako is right and the network effects are overrated. me: Your thoughts and compare Cloud, IaaS, PaaS, SaaSS? To what should the FLOSS world pay more attention and energy? pabs: Initially I dismissed these as buzzwords and a threat to Free Software. These days I view them as potential opportunities for Free Software. Cloud-related technologies such as OpenStack and virtual machines can make private compute farm hardware more flexible and useful to their owners. IaaS providers can be used to run Debian more simply and cheaply and therefore bring Debian to more people than possible with hardware. PaaS providers can be used to run Free Software services. SaaSS can be based entirely on Free Software and respect users. Of course, just like running Free Software on hardware (proprietary or libre), cloud technology, IaaS, PaaS and SaaSS all come with downsides. The FLOSS world should aim to inform users of our software of these downsides. For example, the Debian installer could note that it is running on Intel CPUs with a proprietary BIOS and various proprietary software running, that it is running on a mobile phone with a locked bootloader, that it is running in a Xen VM on machines owned by Amazon. Free Software services could note they are running on Google App Engine etc. Free Software web browsers, chat clients etc could note when they are connecting to proprietary network services. All these notes could inform users about the downsides present in the particular situation encountered. There is also much work to be done making it easier to run Free Software on top of or use Free Software to connect to all manner of platforms from lowRISC to UEFI to VMware to Google App Engine to GitHub to Facebook. The more places Free Software can reach, the more people will be exposed to the philosophy behind it and the more potential there is for folks to join the community. While co-option of the FLOSS world is a dangerous certainty, co-option of proprietary platforms might be able to expand the reach of the philosophy behind Free Software. me: Your thoughts on Purism (the open hardware laptop initiative that got recently funded on CrowdSupply)? pabs: I don't know enough about that to comment but personally I am more interested in a laptop based on a libre CPU architecture. The RISC-V ISA and the lowRISC project seems to be one of the more promising possibilities at this point in time. me: Did you watch Citizenfour - comments on it? pabs: I've seen the trailer and look forward to watching it at some point, I read there might be a screening at DebConf15.

15 February 2015

Antonio Terceiro: rmail: reviving upstream maintaince

It is always fun to write new stuff, and be able to show off that shiny new piece of code that just come out of your brilliance and/or restless effort. But the world does not spin based just on shiny things; for free software to continue making the world work, we also need the dusty, and maybe and little rusty, things that keep our systems together. Someone needs to make sure the rust does not take over, and that these venerable but useful pieces of code keep it together as the ecosystem around them evolves. As you know, Someone is probably the busiest person there is, so often you will have to take Someone s job for yourself. rmail is a Ruby library able to parse, modify, and generate MIME mail messages. While handling transitions of Ruby interpreters in Debian, it was one of the packages we always had to fix for new Ruby versions, to the point where the Debian package has accumulated quite a few patches. The situation became ridiculous. It was considered to maybe drop it from the Debian archive, but dropping it would mean either also dropping feed2imap and sup or porting both to other mail library. Since doing this type of port is always painful, I decided instead to do something about the sorry state in which rmail was on the upstream side. The reasons why it was not properly maintained upstream does not matter: people lose interest, move on to other projects, are not active users anymore; that is normal in free software projects, and instead of blaming upstream maintainers in any way we need to thank them for writing us free software in the first place, and step up to fix the stuff we use. I got in touch with the people listed as owner for the package on rubygems.org, and got owner permission, which means I can now publish new versions myself. With that, I cloned the repository where the original author had imported the latest code uploaded to rubygems and had started to receive contributions, but that repository was inactive for more than one year. It had already got some contributions from the sup developers which never made it in a new rmail release, so the sup people started using their own fork called rmail-sup . Already in my repository, I have imported all the patches that still made sense from the Debian repository, did a bunch of updates, mainly to modernize the build system, and did a 1.1.0 release to rubygems.org. This release is pretty much compatible with 1.0.0, but since I did not test it with Ruby versions older than than one in my work laptop (2.1.5), I bumped the minor version number as warning to prospective users still on older Ruby versions. In this release, the test suite passes 100% clean, what always gives my mind a lot of comfort:

$ rake
/usr/bin/ruby2.1 -I"lib:." -I"/usr/lib/ruby/vendor_ruby" "/usr/lib/ruby/vendor_ruby/rake/rake_test_loader.rb" "test/test*.rb"
Loaded suite /usr/lib/ruby/vendor_ruby/rake/rake_test_loader
Started
...............................................................................
...............................................................................
........
Finished in 2.096916712 seconds.
166 tests, 24213 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
79.16 tests/s, 11546.95 assertions/s

And in the new release I have just uploaded to the Debian experimental suite (1.1.0-1), I was able to drop all of the patches and just use the upstream source as is. So that s it: if you use rmail for anything, consider testing version 1.1.0-1 from Debian experimental, or 1.1.0 from rubygems.org if you into that, and report any bugs to the [github repository](https://github.com/terceiro/rmail). My only commitment for now is keep it working, but if you want to add new features I will definitively review and merge them.

10 November 2014

Neil Williams: On getting NEW packages into stable

There s a lot of discussion / moaning /arguing at this time, so I thought I d post something about how LAVA got into Debian Jessie, the work involved and the lessons I ve learnt. Hopefully, it will help someone avoid the disappointment of having their package missing the migration into a future stable release. This was going to be a talk at the Minidebconf-uk in Cambridge but I decided to put this out as a permanent blog entry in the hope that it will be a useful reference for the future, not just Jessie. Context LAVA relies on a number of dependencies which were at the time all this started NEW to Debian as well as many others already in Debian. I d been running LAVA using packages on my own system for a few months before the packages were ready for use on the main servers (I never actually installed LAVA using the old virtualenv method on my own systems, except in a VM). I did do quite a lot of this on my own but I also had a team supporting the effort and valuing the benefits of moving to a packaged system. At the time, LAVA was based on Ubuntu (12.04 LTS Precise Pangolin) and a new Ubuntu LTS was close (Trusty Tahr 14.04) but I started work on this in 2013. By the time my packages were ready for general usage, it was winter 2013 and much too close to get anything into Ubuntu in time for Trusty. So I started a local repo using space provided by Linaro. At the same time, I started uploading the dependencies to Debian. json-schema-validator, django-testscenarios and others arrived in April and May 2014. (Trusty was released in April). LAVA arrived in NEW in May, being accepted into unstable at the end of June. LAVA arrived in testing for the first time in July 2014. Upstream development continued apace and a regular monthly upload, with some hotfixes in between, continued until close to the freeze. At this point, note that although upstream is a medium sized team, the Debian packaging also has a team but all the uploads were made by me. I planned ahead. I knew that I would be going to Macau for Linaro Connect in February a critical stage in the finalisation of the packages and the migration of existing instances from the old methods. I knew that I would be on vacation from August through to the end of September 2014 including at least two weeks with absolutely no connectivity of any kind. Right at this time, Django1.7 arrived in experimental with the intent to go into unstable and hence into Jessie. This was a headache for me, I initially sought to delay the migration until after Jessie. However, we discussed it upstream, allocated time within the busy schedule and also sought help from within Debian with the RFH tag. Rapha l Hertzog contributed patches for django1.7 support and we worked on those patches upstream, once I was back from vacation. (The final week of my vacation was a work conference, so we had everyone together at one hacking table.) Still there was more to do, the django1.7 patches allowed the unit tests to complete but broke other parts of the lava-server package and needed subsequent tweaks and fixes. Even with all this, the auto-removal from testing for packages affected by RC bugs in their dependencies became very important to monitor (it still is). It would be useful if some packages had less complex dependency chains (I m looking at you, uwsgi) as the auto-removal also covers build-depends. This led to some more headaches with libmatheval. I m not good with functional programming languages, I did have some exposure to Scheme when working on Gnucash upstream but it wasn t pleasant. The thought of fixing a scheme problem in the test suite of libmatheval was daunting. Again though, asking for help, I found people in the upstream team who wanted to refresh their use of scheme and were able to help out. The fix migrated into testing in October. Just for added complications, lava-server gained a few RC bugs of it s own during that time too fixed upstream but awkward nonetheless. Achievement unlocked So that s how a complex package like lava-server gets into stable. With a lot of help. The main problem with top-level packages like this is the sheer weight of the dependency chain. Something seemingly unrelated (like libmatheval) can seriously derail the migrations. The package doesn t use the matheval support provided by uwsgi. The bug in matheval wasn t in the parts of matheval used by uwsgi. It wasn t in a language I am at all comfortable in fixing but it s my name on the changelog of the NMU. That happened because I asked for help. OK, when django1.7 was scheduled to arrive in Debian unstable and I knew that lava was not ready, I reacted out of fear and anxiety. However, I sought help, help was provided and that help was enough to get upstream to a point where the side-effects of the required changes could be fixed. Maintaining a top-level package in Debian is becoming more like maintaining a core package in Debian and that is a good thing. When your package has a lot of dependencies, those dependencies become part of the maintenance workload of your package. It doesn t matter if those are install time dependencies, build dependencies or reverse dependencies. It doesn t actually matter if the issues in those packages are in languages you would personally wish to be expunged from the archive. It becomes your problem but not yours alone. Debian has a lot of flames right now and Enrico encouraged us to look at what else is actually happening in Debian besides those arguments. Well, on top of all this with lava, I also did what I could to help the arm64 port along and I m very happy that this has been accepted into Jessie as an official release architecture. That s a much bigger story than LAVA yet LAVA was and remains instrumental in how arm64 gained the support in the kernel and various upstreams which allowed patches to be accepted and fixes to be incorporated into Debian packages. So a roll call of helpers who may otherwise not have been recognised via changelogs, in no particular order: Also general thanks to the Debian FTP and Release teams. Lessons learnt
  1. Allow time! None of the deadlines or timings involved in this entire process were hidden or unexpected. NEW always takes a finite but fairly lengthy amount of time but that was the only timeframe with any amount of uncertainty. That is actually a benefit it reminds you that this entire process is going to take a significant amount of time and the only loser if you try to rush it is going to be you and your package. Plan for the time and be sceptical about how much time is actually required.
  2. Ask for help! Everyone in Debian is a volunteer. Yes, the upstream for this project is a team of developers paid to work on this code (and largely only this code) but the upstream also has priorities, requirements, objectives and deadlines. It s no good expecting upstream to do everything. It s no good leaving upstream insufficient time to fit the required work into the existing upstream schedules. So ask for help within upstream and within Debian ask for help wherever you can. You don t know who may be able to help you until you ask. Be clear when asking for help how would someone test their proposed fix? Exactly what are you asking for help doing? (Hint: everything is not a good answer.)
  3. Keep on top of announcements and changes. The release team in Debian have made the timetable strict and have published regular updates, guidelines and status notes. As maintainer, it is your responsibility to keep up with those changes and make others in the upstream team aware of the changes and the implications. Upstream will rely on you to provide accurate information about these requirements. This is almost more important than actually providing the uploads or fixes. Without keeping people informed, even asking for help can turn out to be counter-productive. Communicate within Debian too talk to the teams, send status updates to bugs (even if the status is tag 123456 + help).
  4. Be realistic! Life happens around us, things change, personal timetables get torn up. Time for voluntary activity can appear and disappear (it tends to disappear far more often than extends, so take that into account too).
  5. Do not expect others to do the work for you asking for help is one thing, leaving the work to others is quite another. No complaining to the release team that they are blocking your work and avoid pleading or arguing when a decision is made. The policies and procedures within Debian are generally clear and there are quite enough arguments without adding more. Read the policies, read the guidelines, watch how other packages and other maintainers are handled and avoid those mistakes. Make it easy for others to help deliver what you want.
  6. Get to know your dependency chain follow the links on the packages.debian.org pages and get a handle on which packages are relevant to your package. Subscribe to the bug pages for some of the more high-risk packages. There are tools to help. rc-alert can help you spot problems with runtime dependencies (you do have your own package installed on a system running unstable if not, get that running NOW). Watching build-dependencies is more difficult, especially build-dependencies of a runtime dependency, so watch the RC bug lists for packages in your dependency chain.
Above all else, remember why you and upstream want the packages in Debian in the first place. Debian is a respected distribution and has an acknowledged reputation for stability and portability. The very qualities that you and your upstream desire from having your package in Debian have direct implications for the amount of work and the amount of time that will be required to get your packages into Debian and keep them there. Having your package in Debian will bring considerable benefits but you will be required to invest a considerable amount of time. It is this contribution which is valuable to Debian and it is this work which will deliver the benefits you seek. Being an expert in the one package is wildly inadequate. Debian is about the system, the whole distribution and sooner or later, you as the maintainer will be absolutely required to handle something which is so far out of your comfort zone it s untrue. The reality is that you are not expected to fix that problem you are expected to handle that problem and that includes seeking and acknowledging the help of others. The story isn t over until release day. Having your package in testing the day before the freeze is one step. It may be a large step, but it is only one. The status of that package still needs monitoring. That long dependency chain can still come back and bite. Don t wait for problems to surprise you. Finally One thing I do ask is that other upstream teams and maintainers think about the dependency chain they are creating. It may sound nice to have bindings for every interpreted language possible when releasing your compiled library but it does not help people using that library. Yes, it is more work releasing the bindings separately because a stable API is going to be needed to allow version 1.2.3 to work alongside 1.2.2 and 1.3.0 or the entire effort is pointless. Consider how your upstream package migrates. Consider how adding yet another build-dependency for an optional component makes things exponentially harder for those who need to rely on that upstream. If it is truly optional, release it separately and keep backwards compatibility on each side. It is more work but in reality, all that is happening is that the work is being transferred from the distribution (where it helps only that one distribution and causes duplication into other distributions) into the upstream (where it helps all distributions). Think carefully about what constitutes core functionality and release the rest separately. Combining bindings for php, ruby, python, java, lua and xslt into a single upstream release tarball is a complete nonsense. It simply means that the package gets blocked from new uploads by the constant churn of being involved in every transition that occurs in the distribution. There is a very real risk that the package will miss a stable release simply by having fingers in too many pies. That hurts not only this upstream but every upstream trying to use any part of your code. Every developer likes to think that people are using and benefiting from their effort. It s not nice to directly harm the interests of other developers trying to use your code. It is not enough for the binary packages to be discrete migrations happen by source package and the released tarball needs to not include the optional bindings. It must be this way because it is the source package which determines whether version 1.2.3 of plugin foo can work with version 1.2.0 of the library as well as with version 1.3.0. Maintainers regularly deal with these issues so talk to your upstream teams and explain why this is important to that particular team. Help other maintainers use your code and help make it easier to make a stable release of Debian. The quicker the freeze & release process becomes, the quicker new upstream versions can be uploaded and backported.

8 November 2014

Gunnar Wolf: Collective pain

The following text is not mine. I'm copy-translating a text a dear friend of mine just wrote in Spanish, in Facebook. He writes far better than I do (much better than most people I have known). I am not also a great translator. If you can read Spanish, go read the original.
I hate my country. I want to get the hell out of here. This country stinks. Phrases that appear in talks between Mexicans since yesterday. On the network and outside of it. And to tell the truth, I would have put them between quotation marks if I had not thought them as well. At some point. Because that is the edtent of the pain. Enuogh to hate, to insult, to give up. But we talk and write without realizing that it might be the most terrible thing in all this mess. That the pain makes us give up and consent to play a role in the game that they, the executioners, would pleasedly look at from their tribunes, laughing at us while they hand each other the popcorn. That would be over the line. So lets not give them that joy. Because they surely don't realize we have the obligation to notice it from the very beginning and do something to avoid falling there: The root of the pain they caused us yesterday is because that's how the annihilation of hope feels like. The shout "Alive they were taken" they do not realize but we do is a shout of hope. A pronouncement for the possible goodness in the human being. A testimony of hope in the future. A bet for life. And with his cold address, the federal attorney yesterday wanted to finish the killing of our already aching hope. We cannot grant him that joy. They say it's the last thing that dies. I'd say it's the only thing that should not die. Ever. It finishes and everything finishes. There is no possible justice for the parents of the 43. Much less for the 43. Not even however much the official discourse wants to gets us dizzy with the propaganda saying "we will not rest until". Not even if the president quits that would bring back to their classrooms even one of those that by today are just ashes. And sadly, that's the excuse that man wields to not stop boarding his plane and travel wherever he pleases. The farthest from Mexico, the better. Lets not do the same. Lets remind the world this country is full of us, not of them. That the face of a persn is not the dirtyness on his forehead and cheeks, but the skin that's below, that feels and throbs. Lets show the world Mexico is more the verse than the blood, more the idea than the terror. And to them... Lets not give them the joy. To them, lets make them see that, however hard they try, there are things they will never take from us. Our love for this country, for example. The country, over all things.
- Antonio Malpica. After what appears to be the bitter and sadly expected end of a sad, terrible, unbelievable collective social rupture we have lived for ~50 days. And what comes next? How can it come? How can we expect it? I have no way to answer. We, the country's people, are broken.

27 September 2014

DebConf team: Wrapping up DebConf14 (Posted by Paul Wise, Donald Norwood)

The annual Debian developer meeting took place in Portland, Oregon, 23 to 31 August 2014. DebConf14 attendees participated in talks, discussions, workshops and programming sessions. Video teams captured a lot of the main talks and discussions for streaming for interactive attendees and for the Debian video archive. Between the video, presentations, and handouts the coverage came from the attendees in blogs, posts, and project updates. We ve gathered a few articles for your reading pleasure: Gregor Herrmann and a few members of the Debian Perl group had an informal unofficial pkg-perl micro-sprint and were very productive. Vincent Sanders shared an inspired gift in the form of a plaque given to Russ Allbery in thanks for his tireless work of keeping sanity in the Debian mailing lists. Pictures of the plaque and design scheme are linked in the post. Vincent also shared his experiences of the conference and hopes the organisers have recovered. Noah Meyerhans adventuring to Debian by train, (Inter)netted some interesting IPv6 data for future road and railwarriors. Hideki Yamane sent a gentle reminder for English speakers to speak more slowly. Daniel Pocock posted of GSoC talks at DebConf14, highlights include the Java Project Dependency Builder and the WebRTC JSCommunicator. Thomas Goirand gives us some insight into a working task list of accomplishments and projects he was able to complete at DebConf14, from the OpenStack discussion to tasksel talks, and completion of some things started last year at DebConf13. Antonio Terceiro blogged about debci and the Debian Continuous Integration project, Ruby, Redmine, and Noosfero. His post also shares the atmosphere of being able to interact directly with peers once a year. Stefano Zacchiroli blogged about a talk he did on debsources which now has its own HACKING file. Juliana Louback penned: DebConf 2014 and How I Became a Debian Contributor. Elizabeth Krumbach Joseph s in-depth summary of DebConf14 is a great read. She discussed Debian Validation & CI, debci and the Continuous Integration project, Automated Validation in Debian using LAVA, and Outsourcing webapp maintenance. Lucas Nussbaum by way of a blog post releases the very first version of Debian Trivia modelled after the TCP/IP Drinking Game. Fran ois Marier s shares additional information and further discussion on Outsourcing your webapp maintenance to Debian. Joachim Breitner gave a talk on Haskell and Debian, created a new tool for binNMUs for Haskell packages which runs via cron job. The output is available for Haskell and for OCaml, and he still had a small amount of time to go dancing. Jaldhar Harshad Vyas was not able to attend DebConf this year, but he did tune in to the videos made available by the video team and gives an insightful viewpoint to what was being seen. J r my Bobbio posted about Reproducible builds in Debian in his recap of DebConf14. One of the topics at hand involved defining a canonical path where packages must be built and a BOF discussion on reproducible builds from where the conversation moved to discussions in both Octave and Groff. New helpers dh_fixmtimes and dh_genbuildinfo were added to BTS. The .buildinfo format has been specified on the wiki and reviewed. Lots of work is being done in the project, interested parties can help with the TODO list or join the new IRC channel #debian-reproducible on irc.debian.org. Steve McIntyre posted a Summary from the d-i / debian-cd BoF at DC14, with some of the session video available online. Current jessie D-I needs some help with the testing on less common architectures and languages, and release scheduling could be improved. Future plans: Switching to a GUI by default for jessie, a default desktop and desktop choice, artwork, bug fixes and new architecture support. debian-cd: Things are working well. Improvement discussions are on selecting which images to make I.E. netinst, DVD, et al., debian-cd in progress with http download support, Regular live test builds, Other discussions and questions revolve around which ARM platforms to support, specially-designed images, multi-arch CDs, and cloud-init based images. There is also a call for help as the team needs help with testing, bug-handling, and translations. Holger Levsen reports on feedback about the feedback from his LTS talk at DebConf14. LTS has been perceived well, fits a demand, and people are expecting it to continue; however, this is not without a few issues as Holger explains in greater detail the lacking gatekeeper mechanisms, and how contributions are needed from finance to uploads. In other news the security-tracker is now fixed to know about old stable. Time is short for that fix as once jessie is released the tracker will need to support stable, oldstable which will be wheezy, and oldoldstable. Jonathan McDowell s summary of DebConf14 includes a fair perspective of the host city and the benefits of planning of a good DebConf14 location. He also talks about the need for facetime in the Debian project as it correlates with and improves everyone s ability to work together. DebConf14 also provided the chance to set up a hard time frame for removing older 1024 bit keys from Debian keyrings. Steve McIntyre posted a Summary from the State of the ARM BoF at DebConf14 with updates on the 3 current ports armel, armhf and arm64. armel which targets the ARM EABI soft-float ARMv4t processor may eventually be going away, while armhf which targets the ARM EABI hard-float ARMv7 is doing well as the cross-distro standard. Debian is has moved to a single armmp kernel flavour using Device Tree Blobs and should be able to run on a large range of ARMv7 hardware. The arm64 port recently entered the main archive and it is hoped to release with jessie with 2 official builds hosted at ARM. There is talk of laptop development with an arm64 CPU. Buildds and hardware are mentioned with acknowledgements for donated new machines, Banana Pi boards, and software by way of ARM s DS-5 Development Studio - free for all Debian Developers. Help is needed! Join #debian-arm on irc.debian.org and/or the debian-arm mailing list. There is an upcoming Mini-DebConf in November 2014 hosted by ARM in Cambridge, UK. Tianon Gravi posted about the atmosphere and contrast between an average conference and a DebConf. Joseph Bisch posted about meeting his GSOC mentors, attending and contributing to a keysigning event and did some work on debmetrics which is powering metrics.debian.net. Debmetrics provides a uniform interface for adding, updating, and viewing various metrics concerning Debian. Harlan Lieberman-Berg s DebConf Retrospective shared the feel of DebConf, and detailed some of the work on debugging a build failure, work with the pkg-perl team on a few uploads, and work on a javascript slowdown issue on codeeditor. Ana Guerrero L pez reflected on Ten years contributing to Debian.

15 September 2014

Martin Pitt: autopkgtest 3.5: Reboot support, Perl/Ruby implicit tests

Last week s autopkgtest 3.5 release (in Debian sid and Ubuntu Utopic) brings several new features which I d like to announce. Tests that reboot For testing low-level packages like init or the kernel it is sometimes desirable to reboot the testbed in the middle of a test. For example, I added a new boot_and_services systemd autopkgtest which configures grub to boot with systemd as pid 1, reboots, and then checks that the most important services like lightdm, D-BUS, NetworkManager, and cron come up as expected. (This test will be expanded a lot in the future to cover other areas like the journal, logind, etc.) In a testbed which supports rebooting (currently only QEMU) your test will now find an autopkgtest-reboot command which the test calls with an arbitrary marker string. autopkgtest will then reboot the testbed, save/restore any files it needs to (like the tests file tree or previously created artifacts), and then re-run the test with ADT_REBOOT_MARK=mymarker. The new Reboot during a test section in README.package-tests explains this in detail with an example. Implicit test metadata for similar packages The Debian pkg-perl team recently discussed how to add package tests to the ~ 3.000 Perl packages. For most of these the test metadata looks pretty much the same, so they created a new pkg-perl-autopkgtest package which centralizes the logic. autopkgtest 3.5 now supports an implicit debian/tests/control control file to avoid having to modify several thousand packages with exactly the same file. An initial run already looked quite promising, 65% of the packages pass their tests. There will be a few iterations to identify common failures and fix those in pkg-perl-autopkgtest and autopkgtestitself now. There is still some discussion about how implicit test control files go together with the DEP-8 specification, as other runners like sadt do not support them yet. Most probably we ll declare those packages XS-Testsuite: autopkgtest-pkg-perl instead of the usual autopkgtest. In the same vein, Debian s Ruby maintainer (Antonio Terceiro) added implicit test control support for Ruby packages. We haven t done a mass test run with those yet, but their structure will probably look very similar.

2 September 2014

Antonio Terceiro: DebConf 14: Community, Debian CI, Ruby, Redmine, and Noosfero

This time, for personal reasons I wasn t able to attend the full DebConf, which started on the Saturday August 22nd. I arrived at Portland on the Tuesday the 26th by noon, at the 4th of the conference. Even though I would like to arrive earlier, the loss was alleviated by the work of the amazing DebConf video team. I was able to follow remotely most of the sessions I would like to attend if I were there already. As I will say to everyone, DebConf is for sure the best conference I have ever attended. The technical and philosophical discussions that take place in talks, BoF sessions or even unplanned ad-hoc gathering are deep. The hacking moments where you have a chance to pair with fellow developers, with whom you usually only have contact remotely via IRC or email, are precious. That is all great. But definitively, catching up with old friends, and making new ones, is what makes DebConf so special. Your old friends are your old friends, and meeting them again after so much time is always a pleasure. New friendships will already start with a powerful bond, which is being part of the Debian community. Being only 4 hours behind my home time zone, jetlag wasn t a big problem during the day. However, I was waking up too early in the morning and consequently getting tired very early at night, so I mostly didn t go out after hacklabs were closed at 10PM. Despite all of the discussion, being in the audience for several talks, other social interactions and whatnot, during this DebConf I have managed to do quite some useful work. debci and the Debian Continuous Integration project I gave a talk where I discussed past, present, and future of debci and the Debian Continuous Integration project. The slides are available, as well as the video recording. One thing I want you to take away is that there is a difference between debci and the Debian Continuous Integration project: A few days before DebConf, C dric Boutillier managed to extract gem2deb-test-runner from gem2deb, so that autopkgtest tests can be run against any Ruby package that has tests by running gem2deb-test-runner --autopkgtest. gem2deb-test-runner will do the right thing, make sure that the tests don t use code from the source package, but instead run them against the installed package. Then, right after my talk I was glad to discover that the Perl team is also working on a similar tool that will automate running tests for their packages against the installed package. We agreed that they will send me a whitelist of packages in which we could just call that tool and have it do The Right Thing. We might be talking here about getting autopkgtest support (and consequentially continuous integration) for free for almost 2000 4000 packages. The missing bits for this to happen are: During a few days I have mentored Lucas Kanashiro, who also attended DebConf, on writing a patch to add support for email notifications in debci so maintainers can be pro-actively notified of status changes (pass/fail, fail/pass) in their packages. I have also started hacking on the support for distributed workers, based on the initial work by Martin Pitt: Ruby I had some discusion with Christian about making Rubygems install to $HOME by default when the user is not root. We discussed a few implementation options, and while I don t have a solution yet, we have a better understanding of the potential pitfalls. The Ruby BoF session on Friday produced a few interesting discussions. Some take away point include, but are not limited to: Redmine I was able to make Redmine work with the Rails 4 stack we currently have in unstable/testing. This required using a snapshot of the still unreleased version 3.0 based on the rails-4.1 branch in the upstream Subversion repository as source. I am a little nervous about using a upstream snapshot, though. According to the "roadmap of the project ":http://www.redmine.org/projects/redmine/roadmap the only purpose of the 3.0 release will be to upgrade to Rails 4, but before that happens there should be a 2.6.0 release that is also not released yet. 3.0 should be equivalent to that 2.6.0 version both feature-wise and, specially, bug-wise. The only problem is that we don t know what that 2.6.0 looks like yet. According to the roadmap it seems there is not much left in term of features for 2.6.0, though. The updated package is not in unstable yet, but will be soon. It needs more testing, and a good update to the documentation. Those interested in helping to test Redmine on jessie before the freeze please get in touch with me. Noosfero I gave a lighting talk on Noosfero, a platform for social networking websites I am upstream for. It is a Rails appplication licensed under the AGPLv3, and there are packages for wheezy. You can checkout the slides I used. Video recording is not available yet, but should be soon. That s it. I am looking forward to DebConf 15 at Heidelberg. :-)

11 June 2014

Bits from Debian: Introducing the Debian Continuous Integration project

Debian is a big system. At the time of writing, the unstable distribution has more than 20,000 source packages, building more then 40,000 binary packages on the amd64 architecture. The number of inter-dependencies between binary packages is mind-boggling: the entire dependency graph for the amd64 architecture contains a little more than 375,000 edges. If you want to expand the phrase "package A depends on package B", there are more than 375,000 pairs of packages A and B that can be used. Every one of these dependencies is a potential source of problems. A library changes the semantics of a function call, and then programs using that library that assumed the previous semantics can start to malfunction. A new version of your favorite programming language comes out, and a program written in it no longer works. The number of ways in which things can go wrong goes on and on. With an ecosystem as big as Debian, it is just impossible to stop these problems from happening. What we can do is trying to detect when they happen, and fix them as soon as possible. The Debian Continuous Integration project was created to address exactly this problem. It will continuously run test suites for source packages when any of their dependencies is updated, as well as when a new version of the package itself is uploaded to the unstable distribution. If any problems that can be detected by running an automated test suite arise, package maintainers can be notified in a matter of hours. Antonio Terceiro has posted on his blog an introduction to the project with a more detailed description of the project, its evolution since January 2014 when it was first introduced, an explanation of how the system works, and how maintainers can enable test suites for their packages. You might also want to check the documentation directly.

Next.

Previous.